Article 5
Wisdom of crowds for robust gene network inference That “The wisdom of crowds” is not only useful for sociological dilemmas or research, but can also be applied to natural sciences, is proven in the article “Wisdom of crowds for robust gene network inference”. The article by Marbach et al. (2012) demonstrates this by applying the integration of predictions from multiple expert studies in the field of gene studies on Escherichia coli, Staphylococcus aureus and Saccharomyces cerevisiae. The authors based their research approach on the phenomenon of the wisdom of the crowds, being aware of the strengths of collective knowledge. After the assessment of over thirty gene regulatory network studies, they find that there is no single inference method which performs optimally across all datasets. However, after the integration of predictions from the studies into an aggregated inference method, this leads them to robust and high performance across diverse datasets. The project that was studied in Marbach’s study was the DREAM project, referring to ‘Dialogue for Reverse Engineering Assessments and Methods’. This project is established as a framework to enable assessments through common benchmarks and standardized performance metrics, and the project aims to catalyze the interaction between theory and experiments in systems biology. Each year, a community of experts is invited to infer transcriptional regulatory networks from the datasets, leading to multiple predictions about gene expressions. However, the performance of these different inference methods varies strongly, with each time a different optimal method depending on the contextual factors and the setting. The study from Marbach et al. integrates all the predictions from the different inference methods as proposed by the experts involved in the DREAM project. They find that the community-based consensus networks achieve by far the best overall performance, and are robust across all species and datasets. The main conclusion of the article is that the shortcomings of the individual methods are mitigated by aggregating all inference methods in one overarching methods, indicating that also in the natural sciences the knowledge of communities outperforms individual knowledge. A possible explanation for this is that individual studies that introduce a novel inference methods, tend to overestimate their performance as they tend to focus on their advantages for the particular situation. These advantages of different methods can complement each other when they are aggregated, whereas the limitations can be leveled out. Therefore, integrated prediction for inference methods turns out to be the best strategy. The added value of this paper to the debate of scientific importance of the phenomenon of wisdom of the crowds, is that the concept can be applied to the findings of scientific experts as well. This study shows that aggregating the different prediction methods of systems biologists lead to the most optimal prediction method in science. Therefore, this article underscores the importance of wisdom of the crowds for scientific findings, and implies that it could not only be useful for natural sciences but that it could extend to other fields of research as well. Marbach, D., Costello, J. C., Küffner, R., Vega, N. M., Prill, R. J., Camacho, D. M., ... & DREAM5 Consortium. (2012). Wisdom of crowds for robust gene network inference. Nature methods, 9(8), 796-804.